Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: add consensus to e2e flow test #1811

Merged
merged 1 commit into from
Nov 12, 2024
Merged

Conversation

dan-starkware
Copy link
Collaborator

No description provided.

@reviewable-StarkWare
Copy link

This change is Reviewable

Copy link

github-actions bot commented Nov 4, 2024

Artifacts upload triggered. View details here

Copy link

codecov bot commented Nov 4, 2024

Codecov Report

Attention: Patch coverage is 92.59259% with 2 lines in your changes missing coverage. Please review.

Project coverage is 76.39%. Comparing base (e3165c4) to head (671118d).
Report is 332 commits behind head on main.

Files with missing lines Patch % Lines
...us_orchestrator/src/sequencer_consensus_context.rs 60.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #1811       +/-   ##
===========================================
+ Coverage   40.10%   76.39%   +36.28%     
===========================================
  Files          26      373      +347     
  Lines        1895    39845    +37950     
  Branches     1895    39845    +37950     
===========================================
+ Hits          760    30438    +29678     
- Misses       1100     7138     +6038     
- Partials       35     2269     +2234     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

github-actions bot commented Nov 5, 2024

Artifacts upload triggered. View details here

Copy link
Contributor

@Itay-Tsabary-Starkware Itay-Tsabary-Starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 7 files reviewed, 4 unresolved discussions (waiting on @dan-starkware)


crates/tests-integration/src/integration_test_utils.rs line 72 at r1 (raw file):

}

fn create_consensus_manager_config()

Current name is confusing, as there are other create_X_config that just return a config object, while this variant also returns the channels. Could you please rename?

Code quote:

create_consensus_manager_config

crates/tests-integration/tests/end_to_end_test.rs line 44 at r1 (raw file):

        .expect("listen to broadcasted messages should finish in time");
    });
    join_handle.await.expect("Task should succeed");

Please wrap the "await for something to happen within a timeout" as a utility function (or add a TODO, can include my name as well).

Code quote:

    let join_handle = tokio::spawn(async move {
        tokio::time::timeout(
            LISTEN_TO_BROADCAST_MESSAGES_TIMEOUT,
            listen_to_broadcasted_messages(
                mock_running_system.consensus_proposals_channels,
                &expected_batched_tx_hashes,
            ),
        )
        .await
        .expect("listen to broadcasted messages should finish in time");
    });
    join_handle.await.expect("Task should succeed");

crates/tests-integration/tests/end_to_end_test.rs line 51 at r1 (raw file):

    expected_batched_tx_hashes: &[TransactionHash],
) {
    // TODO(Dan, Guy): retrieve chian ID (from the config?).

Typo

Code quote:

chian

crates/tests-integration/tests/end_to_end_test.rs line 51 at r1 (raw file):

    expected_batched_tx_hashes: &[TransactionHash],
) {
    // TODO(Dan, Guy): retrieve chian ID (from the config?).

I'd suggest modifying IntegrationTestSetup to hold it as a member, and instantiate the value using StorageTestSetup.

Code quote:

(from the config?)

crates/tests-integration/tests/end_to_end_test.rs line 71 at r1 (raw file):

            );
        }
    }

I didn't review this part, I suggest asking someone more familiar with the consensus to do so. If there's no on else I'll dig into this though, please lmk.

Code quote:

    let mut broadcasted_messages_receiver =
        consensus_proposals_channels.broadcasted_messages_receiver;
    let mut received_tx_hashes = vec![];
    while received_tx_hashes.len() < expected_batched_tx_hashes.len() {
        let (message, _broadcasted_message_metadata) = broadcasted_messages_receiver
            .next()
            .await
            .unwrap_or_else(|| panic!("Expected to receive a message from the broadcast topic"));
        if let ProposalPart::Transactions(transactions) = message.unwrap() {
            received_tx_hashes.append(
                &mut transactions
                    .transactions
                    .iter()
                    .map(|tx| tx.calculate_transaction_hash(&chain_id).unwrap())
                    .collect(),
            );
        }
    }

Copy link
Collaborator Author

@dan-starkware dan-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 7 files reviewed, 3 unresolved discussions (waiting on @Itay-Tsabary-Starkware)


crates/tests-integration/src/integration_test_utils.rs line 72 at r1 (raw file):

Previously, Itay-Tsabary-Starkware wrote…

Current name is confusing, as there are other create_X_config that just return a config object, while this variant also returns the channels. Could you please rename?

Done.


crates/tests-integration/tests/end_to_end_test.rs line 44 at r1 (raw file):

Previously, Itay-Tsabary-Starkware wrote…

Please wrap the "await for something to happen within a timeout" as a utility function (or add a TODO, can include my name as well).

Done.


crates/tests-integration/tests/end_to_end_test.rs line 51 at r1 (raw file):

Previously, Itay-Tsabary-Starkware wrote…

I'd suggest modifying IntegrationTestSetup to hold it as a member, and instantiate the value using StorageTestSetup.

Done.

Copy link

github-actions bot commented Nov 5, 2024

Artifacts upload triggered. View details here

Copy link
Contributor

@Itay-Tsabary-Starkware Itay-Tsabary-Starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 5 of 7 files at r1, 2 of 2 files at r2, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @dan-starkware)

Copy link

github-actions bot commented Nov 5, 2024

Artifacts upload triggered. View details here

Copy link

github-actions bot commented Nov 5, 2024

Artifacts upload triggered. View details here

Copy link

github-actions bot commented Nov 7, 2024

Artifacts upload triggered. View details here

Copy link

Artifacts upload triggered. View details here

Copy link

Artifacts upload triggered. View details here

Copy link
Contributor

@matan-starkware matan-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 7 of 7 files at r7, all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @dan-starkware, @guy-starkware, and @Itay-Tsabary-Starkware)


crates/tests-integration/tests/end_to_end_test.rs line 39 at r5 (raw file):

Previously, dan-starkware wrote…

Done.

I don't see any change, which is fine for this PR, just noting since you wrote "Done"


crates/tests-integration/tests/end_to_end_test.rs line 68 at r5 (raw file):

Previously, dan-starkware wrote…

Why? This is a utility for testing.

macros are harder to work with in the IDE, you don't get nice type hints, and jumping to the definition doesn't bring you to the actual function used, still requires more sleuthing. They also make compile times slower.

I prefer functions when there is a 1 to 1 replacement.

Copy link

Artifacts upload triggered. View details here

Copy link
Collaborator Author

@dan-starkware dan-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 5 of 13 files reviewed, 1 unresolved discussion (waiting on @guy-starkware, @Itay-Tsabary-Starkware, and @matan-starkware)


crates/tests-integration/tests/end_to_end_test.rs line 39 at r5 (raw file):

Previously, matan-starkware wrote…

I don't see any change, which is fine for this PR, just noting since you wrote "Done"

I meant ACK

Copy link

Artifacts upload triggered. View details here

Copy link
Contributor

@matan-starkware matan-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 7 of 8 files at r8, 1 of 1 files at r9, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @guy-starkware and @Itay-Tsabary-Starkware)

@dan-starkware dan-starkware force-pushed the dan/e2e/add_consensus branch 2 times, most recently from abdcb32 to 1f044ac Compare November 12, 2024 11:08
Copy link

Artifacts upload triggered. View details here

Copy link
Contributor

@Itay-Tsabary-Starkware Itay-Tsabary-Starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 6 files at r5, 3 of 7 files at r7, 4 of 8 files at r8, 1 of 1 files at r10, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @guy-starkware)

Copy link

Artifacts upload triggered. View details here

Copy link
Contributor

@ShahakShama ShahakShama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 1 of 6 files at r5, 2 of 7 files at r7, 1 of 8 files at r8, 10 of 10 files at r11, all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @dan-starkware and @guy-starkware)

Copy link
Contributor

@ShahakShama ShahakShama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @dan-starkware and @guy-starkware)

Copy link
Contributor

@ShahakShama ShahakShama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @dan-starkware and @guy-starkware)

Copy link
Contributor

@ShahakShama ShahakShama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @graphite-app[bot] and @guy-starkware)

Copy link
Contributor

@yair-starkware yair-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dismissed @graphite-app[bot] from a discussion.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @guy-starkware)

Copy link
Contributor

@yair-starkware yair-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dismissed @guy-starkware from a discussion.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @guy-starkware)

@dan-starkware dan-starkware merged commit 6a1305e into main Nov 12, 2024
20 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Nov 14, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants